Weak and strong regularity, compactness, and 
approximation of polynomials 

Alexander SchrijveiQ 

Abstract. Let X be an inner product space, let G be a group of orthogonal transformations of X, 
and let R be a bounded G-stable subset of X. We define very weak and very strong regularity for 
such pairs (R, G) (in the sense of Szemeredi's regularity lemma), and prove that these two properties 
are equivalent. 

Moreover, these properties are equivalent to the compactness of the space (B(H),dn)/G. Here 
H is the completion of X (a Hilbert space), B(H) is the unit ball in H, da is the metric on H given 
by dji(x,y) := sup reR \(r,x — y)\, and (B(H),dn)/G is the orbit space of (B(H),dn) (the quotient 
topological space with the G-orbits as quotient classes). 

As applications we give Szemeredi's regularity lemma, a related regularity lemma for partitions 
into intervals, and a low rank approximation theorem for homogeneous polynomials. 

1. Equivalence of very weak regularity, very strong regularity, 
and compactness 

This paper is inspired by Szemeredi's regularity lemma ([7]) and subsequent work on graph 
limits by Lovasz and Szegedy ([3,4]) (cf. also [5]). 

Let A be an inner product space and let R be a bounded subset of A spanning X. (So 
each element of A is a linear combination of finitely many elements of R.) Let G be a group 
of orthogonal transformations ir of X with tv(R) = R. Let B{X) denote the unit ball in X. 
For any k, let Rk '■= {=t?T db • • • ±r& | r±, . . . , E R}. Let H be the completion of A, which 
is a Hilbert space. Then G naturally acts on H. For x,y E H, define 

(1) d R (x,y) := sup|(r,x - y)\. 

r<=R 

The space (B(H),cIr)/G is the orbit space of (B(H),dji), i.e., the quotient topological space 
of (B(H),dji) taking the G-orbits as classes. 

Theorem 1. The following are equivalent: 

(i) (R, G) is very weakly regular: for each k there exists a finite set Z C A such that for 
each x E Rk there exist z E Z and tt E G satisfying (r, x — z 77 ) 2 < 1 for each r E R; 

(ii) (R,G) is weakly regular: for each e > there exists a finite set Z C B(X) such that 
for each x E B(X) there exist z E Z and it E G satisfying \ (r,x — z n )\ < e for each r E R; 

(iii) (R, G) is very strongly regular: for each e > and f : X — > {1, 2, . . .} there exists a 
finite set Z C B(X) such that for each x E B(X) there exist z E Z and 7r € G satisfying 
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(2) ^Kn^-^l^ea + [|(||n||*,... A\rmf)\U 

1=1 

for all t E [e, 2], where p := 2/ (2 — t), and for all orthogonal r%, . . . , Tft z \ E R; 
(iv) t/ie space (B(H),d R )/G is compact. 

Proof. (iii)=^(ii) follows by taking f(x) = 1 for each x £ X and t = 1. (ii)=>(i) follows by 
observing that C B(X) for some i, and taking e := 1/1 So it suffices to prove (i)=>(ii), 
(ii)=^(iv), and (iv)=>-(iii). 
For all x,y E B{H) define 

(3) S R (x,y) := inf d R (x, y n ). 

Then 5 R is a pseudometric, and the space (B(H),5 R ) is topologically homeomorphic to the 
orbit space (B(H),d R )/G. 

Observe that (i) implies that the space (R k ,5 R ) is totally bounded^. Indeed, choose 
e > 0. Let t := le- 1 ] . Then R kt can be covered by finitely many 5^-balls of radius 1. As 
Rk ^= jRkti Rk can be covered by finitely many ^-balls of radius 1/t < e. 

So we can assume, by scaling, that ||r|| < 1 for each r E R. 

(i)=>(ii): We saw above that (i) implies that (R k ,5 R ) is totally bounded for each k. Now 
define, for each k, 

(4) S k := {Am H h \ k r k | n, . . . , r k E R, Ai, . . . , X k E [-1,+1]}. 

Then also (S k ,5 R ) is totally bounded. Indeed, choose e > 0, and define t := fc|~e -1 ]. Then 
each x E £& has Hilbert distance less than e to \R k t- By the above, (R k t,$ R ) is totally 
bounded, hence so is (jR k t,S R ). So (S k ,5 R ) is totally bounded. 
Next we show that for each k: 

(5) B(X) C B dR (S k ,l/Vk). 

To see this, choose a E B(X). Let ao := a. If aj has been found, and d^(ai,0) > 1/Vk, 
choose r with (r, ctj) > \j\fk. Let aj+i := — (r, aj)r. Then by induction on i, as ||r|| < 1, 

(6) ll a i+i|| 2 = l|aj|| 2 - 2(r,aj) 2 + (r,aj) 2 ||r|| 2 < ||aj|| 2 - (r,aj) 2 < ||aj|| 2 - 1/k < 
l-i/k-l/k = l- (i + l)/k. 

So the process terminates for some i < k, and we have ([5]), since a — ai E S k and hence 
d R (a, S k ) < d R (a, a - o») = 0) < 

As each (S k ,5 R ) is totally bounded, ([5]) implies that (B(X),5 R ) is totally bounded. 

3 A pseudometric space is totally bounded if for each e > it can be covered by finitely many balls of 
radius e (cf. [1]). 
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(ii)=^(iv): By (ii), the space (B(H),6r) is totally bounded. So it suffices to show that 
(B(H), 5r) is complete. Let x\,X2, ... be a Cauchy sequence in (B(H),5r). We show that 
it is convergent. We can assume that 5n(x n , x n+ i) < 2~ n for each n. Let tti be the identity 
in G. For each n > 1, we can choose ir n+ i £ G such that dft(x^ n , x^j 1 ) < 2~ n . Replacing 
x n by x^ n , we can assume that xi,X2, ... is a Cauchy sequence in (B(H),dn). As B(H) is 
weakly compact, x\,X2, ■ ■ ■ has a subsequence that converges to some a G B(H) in the weak 
topology on B(H). Then lhm^oo dji(x n , a) = 0. Indeed, dji(x n ,a) < 2~ n+2 for each n. 
Otherwise, | (r, x n — a) \ > 2~ n+2 for some r £ R. As a is weak limit of some subsequence of 
Xi,X2, . . there is an m > n with \(r, x m — a}\ < 2~ n+1 . As \{r, x n — x m ) \ < dn(x n , x m ) < 
2~ n+1 , this gives a contradiction. 

(iv)=>(iii): Choose e > and / : X — )■ {1,2,...}. For any /c, consider the function <^ : 
A R defined by 



(7) fe (x) := sup sup 



tG[e,2] orthogonal 

(! + IKIIn.ll*, • • • , IWI*)|| P ) 

for x G A, where p = (1 — i/2) -1 . Then 0& is continuous with respect to the <iR-topology 
on B(H). To see this, let x,y G with d R (x,y) < 1. Then |(r,x)|* - \{r,y)\ l < 

2|(r, x — y)\ e < 2dfi(x,y) e for each r £ R and t £ [e, 2]Jj" This gives, by considering any 
t and ri, . . . ,rfc in the suprema for x, that > (f>k(%) — 2kdn(x,y) e (using that the 

denominator in ([7]) is at least 1). So fa is continuous in the d^-topology on B(H). 
Define for each z G B(X): 

(8) U s := {x € B(H) \ <t> f(z) (x - z) < e}. 

So U z is open in de d#-topology. Moreover, the U z for z G B(X) cover B(H). Indeed, for 
any a; G B(H) there exists z G B(X) with ||x — z|| < e 1//e . Then x G E/j, since <^fc(x — z) < e 
for any k, which follows from the following inequality. Let t G [e, 2] and r%, . . . , G i? be 
orthogonal and nonzero, for some A; > 1. Define Si := rj/||rj|| for each i. So si,...,s& are 
orthonormal. Denote p := \\(\\ri\\ t , ■ ■ ■ , ||^fe||*)||p, with p := 2/(2 — i). Then one has for any 
y G B(H), using the Holder inequality, and setting q := 2/t (so that p _1 + q^ 1 = 1): 

fc k k k 



(9) £Kr^>|' = £ INI* • |<«*,y>l* < (E H^ll ip ) 1/P • E K-i. v)!* 9 ) 179 

i=l i=l i=l i=l 

k 

p(£(si, y) 2 ) 1 '" < P \\y\\ 2/q = pWyW* < (1 + 



i=l 



So (/>fr z \(x — z) < \\x — z\\ £ < e, and hence x G U z . 

As (B(H), 5r) is compact by (iv), there is a finite set Z C X such that voor each x G A 
there exist z £ Z and ir £ G such that x G fT^*-. This gives (iii). I 



4 This follows from the fact that if < b < a < 1, then for each t £ [1, 2]: a* - b l < a 1 - b l + (a 2 ~* - 
6 2 "')(a6)' _1 = (a - &)(a t_1 + fa'" 1 ) < 2(o - 6) < 2(a - b) £ , and for each t £ [e, 1), by the concavity of the 
function x': a* — 6* < (a — 6)' < (a — by. 
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2. Applications 



Since R spans X, X is fully determined by the positive semidefinite R x R matrix giving 
the inner products of pairs from R. Then G is given by a group of permutations of R that 
leave the matrix invariant. It is convenient to realize that R is weakly regular if (but not 
only if) the orbit space R k /G is compact for each k. 

1. Szemeredi's regularity lemma [7]. Let R be the collection of sets I x J, with / and 
J each being a union of finitely many subintervals of [0, 1], with inner product equal to the 
measure of the intersection. Let G be the group of permutations of the intervals of any 
partition of [0, 1] into intervals. Then G acts on R. 

Let II be the collection of partitions of [0, 1] into finitely many sets, each being a union 
of finitely many intervals. For P, Q G II, P < Q if and only if P is a refinement of Q. This 
gives a lattice; let A be the meet. 

For any P G II, let Lp be subspace of X spanned by the elements I x J with /, J G P. 
For any x £ X, let xp be the orthogonal projection of x onto Lp. 

Lemma 1. For each x G X and e > there exists t £)X such that for each N G II there is a 
P > N such that ||xjv — xp\\ < e and \P\ < t £tX . 

Proof. Let Y be the set of those x for which the statement holds for all e > 0. Then Y is 
a linear space. Indeed, if x G Y and A / then Ax G Y, as we can take t £: \ x := t|^-i| E)X . 
If x, y G Y then x + y G Y, as we can take t £tX+y := t £ /2 tX t £ /2, y , since if \\xn — xp\\ < 
e/2 and ||j/jv — vq\ < e/2 for some P,Q > N, then \\(x + j/)jv — xp — j/q|| < e, hence 
II (x + y)N ~(x + v)paq\\ < £, since x P + y Q G L PAQ and ((x + y)N)pAQ = (x + v)paq (since 
Lp/\Q C Ljv). Note that |P A Q| < |P||Q|. 

So y is a linear space, and hence it suffices to show that BCY. Let x £ R and e > 0. 
We claim that t £)X := (1 + 2/e) 2 will do. Indeed, let N G II. Then 

(10) x N = x J ) 

i,JeN 

for some a, (3 : N — > [0, 1]. Let a' and be obtained from a and /3 by rounding down the 
values to an integer multiple of e/2. Let P > N be such that two classes I and J of iV are 
contained in the same class of P if and only if a'j = a'j and j3j = As the pairs (a'^P'j) 
take at most (1 + 2/e) 2 different values, we have \P\ < (1 + 2e~ 1 ) 2 . Define 

(11) y := x J). 

i,JeN 

Then y G Lp. Hence, since xp = (xjv)p (as Lp C Ljv), implying that xp is the point on 
Lp closest to xat: 

(12) ||xtv-xp|| 2 < ||x7v-y|| 2 < {a^j-a'jp'jf^Ix J) < e 2 ^i{IxJ)=e 2 . 

i,JeN i,JeN 
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Here x J) is the measure of / x J. 
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Call a collection P of sets balanced if all sets in P have the same cardinality. Call 
a partition P of a finite set V e -balanced if P \ P' is balanced for some P' C P with 

iu^i<em 

Lemma 2. Let e > 0. T/ten eac/i partition P of a finite set V has an e -balanced refinement 
Q with \Q\ < (1 + l/e)|P|. 

Proof. Define t := e|V|/|P|. Split each class of P into classes, each of size \t], except for 
at most one of size less than t. This gives Q. Then \Q\ < \P\ + \V\/t = (1 + l/e)|P|. 
Moreover, the union of the classes of Q of size less than t has size at most \P\t = e\V\. So 
Q is e-balanced. I 

Given a graph H = (V, E) and C,D QV, then e(C, D) is the number of adjacent pairs 
of vertices inCxD. J£C,D^ 0, let d(C,D) := e(C, D)/\C\\D\. 

Theorem 2 (Szemeredi's regularity lemma). For each e > andp £ N there exists k P)£ £ N 
such that for each graph H = (V, E) and each partition P of V with \P\ = p there is an 
e-balanced refinement Q of P with \Q\ < k Pj£ and 

(13) Yl $ fBx(\C\\D\-\d(C,D)-d(AxB)\f<s\V\ 2 . 

Proof. Let R and G be as above. It is easy to check that R k /G is compact for each k, 
hence (R,G) is very weakly regular. So, by Theorem (R,G) is very strongly regular. 

Fix e > and p £ N. For each x G X, define f(x) := ((1 + f/s)pt e /^ x ) 2 -, where t e i^ x is 
as given in Lemma [TJ 

By the very strong regularity of (P, G) , there exists a finite set Z C X such that for 
each x G B(X) there exist z £ Z and ir £ G satisfying 

/« 

(14) y^(rj,a; - z 7 ") 2 < e 2 /16 for all orthogonal n, . . . ,?7( z ) G ^2- 
i=i 

Let := max{/(z) | z G Z}. We show that k Py£ is as required. 

Let H = ([n], -E7) be a graph. Let iV be the partition of [0, 1] into n equal consecutive 
intervals h,... ,I n , and let x := Sije[n] adjacent h x ^ ( tne corresponding graphon). 

By the above there exists & z £ Z and a ir £ G satisfying p4p . By Lemma [H there is a 
partition U G II with U > N such that |C/| < t e / 4>z and ||^jv - < e/4. Let 8 := P AU. 
So |5| < |P||C/| < pt e /^ z . By Lemma [21 there is an e-balanced refinement Q of 5 with 
2V < Q < S and |Q| < (1 + l/e)|5| < y/f(z) < k PjE . We show that this Q gives the 
partition of the theorem. 

For each A, B £ Q, choose r £ R with r C A x B, such that r G L^v and such that 
\{r, x — zq)\ is maximized. This implies for each r' £ R with r' C i x B and r' £ L^: 

(15) \(r',x-x Q )\ < \(r',x-z Q )\ + \(r',x Q -z Q }\ < |(r', x- zq)\ + |(A x B, xq- zq)\ = 
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\(r',x- z Q )\ + \(A x B,x-zq)\ < 2\{r,x-z Q )\. 

Let ri, . . . ,r t be the chosen elements. So t = \Q\ 2 < f(z). Hence, noting that (rj,z) = 
(n,ZN), since n £ Ln, 

t t 

(16) £>,x - zq) 2 ) 1 / 2 < x - z N ) 2 )^ 2 + \\z N - z Q \\ < 

i=l i=l 
t 

(^(r l ,x-z) 2 ) 1 / 2 + e/4<e/2. 
i=l 

For the graph ff, CE]) and (US]) give (H3|). | 

To interpret (|13p . for A,B £ Q, let tua,b denote the maximum described in (JT3J). Let 
Q' be such that Q \ Q' is balanced and | (J Q'\ < e\V\. Set Q" :=Q\ Q', and let Z be the 
collection of pairs (A,B) G Q" x Q" with m A)B > ^/e\A\\B\. Then CC| implies 

(17) ^ |A||5|< z- 1/2 mA,B<V^\V\ 2 . 
(A,B)ez (A,B)ez 

Moreover, as | |J Q'\ < e\V\, 

(is) \ A \\ B \^ J2 \ A \\ B \ - 2e \ y \ 2 = ( i - 2e )\ v \ 2 - 

A,BgQ" A,BeQ 

Hence, assuming e < 1/4, \Z\ < y/e(l - 2e)- l \Q"\ 2 < 2^e\Q"\ 2 . For each (A,B) G 
(Q" x Q") \ Z one has vtla^b < -^e\A\\B\, implying that for each rectangle R C A x B with 
\R\/\A x B\ > tfe one has \d(R) - d(A x 2?)| < tfe. In other words, A x B is ^/e-regular. 

2. "Interval regularity". Let R be the collection of sets I x J, with I and J subintervals 
of [0, 1], with inner product given by the measure of the intersection. Then Theorem Q] gives 
an "interval regularity theorem" for graphs (it can also be proved with Szemeredi's classical 
combinatorial method): 

Theorem 3. For each e > andp G N there exists /c P)£ G N such that for each n, each graph 
H = {[n],E) and each partition P of [n] into intervals with \P\ < p, P has a refinement 
to a partition Q into at most k p ^ £ intervals such that all intervals in Q have the same size 
except for some of them covering < en vertices and such that 

( 19 ) E \I\\AW,J)-d(A,B)\ <en 2 . 

A,BdQ I, J intervals 

Here d(I, J) and d(A, B) are the densities of the corresponding subgraphs of H. 

This can be derived similarly as (in fact, easier than) Szemeredi's regularity lemma 
above. 



6 



3. Polynomial approximation. Let k < n. Each polynomial p G R[xi, . . . , x n ] can 
be uniquely written as p = VJ^ I^P/j,, where fi ranges over the set M of all monomials in 
. . . , Xk] and where p^ G . . . , x n ]. If p is homogeneous of degree d, we say that 

p is e- concentrated on the first k variables if 



(20) y max p^{x) 2 < e\\p 



MeM 11*11 

dcg(M)<d 



where ||p|| is the square root of the sum of the squares of the coefficients of p. 

Theorem 4. For each e > and d G N ifoere exists k^^ such that for each n, each 
homogeneous polynomial of degree d inn variables is e- concentrated on the first k variables 
after some orthogonal transformation of W 1 , for some k < k^ £ - 

This can be derived by setting R to be the set of all polynomials (a T x) d , with a G M. n 
and ||a|| = 1 for some n (setting x = (x\, X2, ■ ■ ■)), taking the inner product of (a T x) d and 
(b T x) d equal to (a T b) d . (This corollary strengthens a 'weak regularity' result of Fernandez 
de la Vega, Kannan, Karpinski, and Vempala [2].) For details, we refer to [6]. 
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